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ABSTRACT 


The  National  Science  Foundation  established  the  Decision  and 
Management  Sciences  Program  (DMS)  in  1982.   Healthy  and  growing,  DMS  is  likely 
to  affect  OR/MS  significantly,  not  only  by  providing  funds  for  basic  research, 
but  also  through  its  vision  of  a  combined  theoretical  and  empirical  science  of 
operational  and  managerial  processes,  and  by  its  policy  of  bringing  together 
different  disciplines  within  the  program.   An  NSF  workshop  held  in  Dallas  in 
April  1984  sought  to  identify  research  opportunities  in  the  decision  and 
management  sciences.   Promising  areas,  many  of  them  inviting  cross- 
disciplinary  reasearch,  appear  in  the  development  of  measurement-based  models 
for  operational  processes  (both  natural  and  designed  and  including 
organizational  and  managerial  activities),  choice  theory  (individual  and  group 
choice,  values,  judgement,  and  risk  behavior),  decision  support,  and  the 
treatment  of  complexity.   Increased  knowledge  in  any  of  these  areas  would  be 
likely  to  have  a  valuable  long  run  impact  on  management  practice. 
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In  1982,  with  strong  endorsement  from  the  professional  societies 
TIMS,  ORSA,  and  AIDS,  the  National  Science  Foundation  established  the  Decision 
and  Management  Science  Program  (DMS) .   During  1982  and  1983  the  program 
started  up  with  a  modest  but  growing  budget  and  awarded  an  initial  series  of 
grants.   In  April  1984,  to  develop  the  program  further  and  to  understand 
better  its  scientific  potential,  NSF  sponsored  a  workshop  at  the  University  of 
Texas  at  Dallas  under  Frank  Bass'  chairmanship.   The  purpose  was  to  identify 
research  opportunities  in  DMS  and  make  recommendations  to  NSF  about  directions 
for  the  program.   This  paper  provides  a  summary  of  the  research  opportunities 
generated  by  the  conference. 

My  belief  is  that  NSF's  Decision  and  Management  Science  program  will 
substantially  affect  the  evolution  of  our  field.   This  will  happen  in  part,  of 
course,  by  the  injection  of  funds  for  basic  research,  but  even  more  because  of 
the  vision  of  the  DMS  charter  and  NSF's  policy  of  deliberately  bringing 
together  different  disciplines  within  the  program. 

DMS  Goals 

A  statement  of  DMS  research  objectives  was  written  by  its  Advisory 
Subpanel  early  in  the  program's  history.   The  statement  emphasizes  the 
creation  of  a  theoretical  and  empirical  science  of  managerial  and  operational 
processes.   The  processes  are  typically  to  be  described  by  mathematical  models 
derived  from  empirical  observations  or  from  a  theory  that  is  subject  to 
empirical  verification.   To  quote  from  the  mission  statement: 

"Thus,  the  body  of  research  supported  by  the  program  should  possess 
generality,  be  based  on  empirical  observation  or  be  subject  to  empirical 
validation,  and  incorporate  social  and  behavioral  aspects.   Processes  should 
be  characterized  by  models  that  are  tested  in  operational  contexts.   Even 
though  an  individual  project  may  not  have  all  these  characteristics,  its 
evolution  toward  this  end  must  be  clear." 

Although  OR/MS  has  done  much  good  in  the  world  over  the  last  30 
years,  the  field  continues  to  fight  a  narrow  image  drawn  largely  from  its  long 
association  with  mathematical  programming,  queuing,  decision  analysis  and 
their  supporting  mathematical  methodologies.   Although  no  one  faults  the 
payoff  from  these  areas,  most  people  in  OR/MS  wish  to  see  the  field  attack 
managerial  and  operational  problems  broadly  and,  in  fact,  tend  to  be 
discontent  with  the  slow  pace  at  which  useful  new  knowledge  is  created  and 
put  into  practice. 

From  this  point  of  view,  the  DMS  statement  serves  as  a  highly 
visible  reaffirmation  of  OR/MS 's  classical  scientific  roots  and  broad 
mission  in  models  and  measurements  for  live  operations.   The  statement, 
reproduced  in  the  Appendix,  also  recognizes  explicitly  the  social  and 
behavioral  aspects  of  mangerial  processes. 

Another  significant  move  by  NSF  has  been  to  involve,  at  the  outset, 
not  only  the  operations  research,  management  science  and  decision  science 
communities  that  were  instrumental  in  establishing  DMS,  but  also  at  least  two 
others  that  are  relevant  but  have  not  been  closely  connected.   One  is  systems 
theory,  which  is  based  primarily  in  electrical  engineering  and  has  many 


members  who  work  on  operational  problems.   The  other,  more  distant  in  culture, 
is  the  branch  of  psychology  that  studies  how  people  actually  make  decisions. 
This  subject  sounds  relevant  to  DMS ,  and  it  is.   The  bringing  of  the  three 
groups  together  establishes  a  broader  disciplinary  base  than  can  be 
constructed  from  the  mainline  OR/MS  community  alone. 

Workshop  on  Research  Priorities 

The  Dallas  workshop  on  DMS  research  priorities  consisted  of  two 
parts,  each  taking  a  day.   The  first  contained  papers  and  presentations:   In 
the  morning  a  series  of  speakers  outlined  their  views  of  high-payoff 
research  opportunities;   in  the  afternoon  people  presented  examples  of 
research  on  operational  phenomena  and  decision  making.   The  second  day  built 
on  these  materials  and  sought  to  identify  research  topics  with  the  potential 
for  significant  advance  in  the  next  5-10  years. 

The  papers  reporting  research  on  operational  phenomena  and  decision 
making  were: 

George  Huber,  "Decision  Support  Systems" 

Richard  Larson,  "Public  Sector  Applications  of  OR" 

Michael  Cohen,  "Computer  Models  and  Organizational  Design" 

Detlof  von  Winterfeldt  and  Ralph  Keeney,  "Value-Focussed  Analysis" 

Alexander  Levis,     "Decision  Making  in  C   Systems" 

Lola  Lopes,  "Psychological  Theories  of  Risk" 

Subrata  Sen,  "Marketing" 

Andrew  Whinston,  "Decision  Support  Systems  and  Organizational 

Complexity" 
Robert  Winkler,  "Acquiring  Information" 
Lofti  Zadeh,  "Expert  Systems  and  Fuzzy  Sets" 

As  may  be  seen  from  the  topics,  at  least  the  three  different  disciplines 
mentioned  earlier  are  represented.   A  selection  of  these  papers  is  appearing 
in  the  IEEE  journal,  SYSTEMS,  MAN  AND  CYBERNETICS,  whose  editor,  Andrew  Sage, 
chaired  the  session,  and  so  they  will  not  be  covered  here. 

The  session  on  high  payoff  research  opportunities  fitting  the  D.MS 
mission,  chaired  by  Alfred  Blumstein,  contained  the  following  presentations: 

Frank  Bass:   Management  Science 

Richard  Larson:   Operations  Research 

Kenneth  Hammond:   Psychology 

Hillel  Einhorn:   Behavioral  Decision  Making 

Edison  Tse:   Systems  Theory 

The  material  from  that  session  along  with  the  entire  second  day  will  be  the 
subject  of  my  summary  on  research  opportunities.   Two  of  the  papers,  Hammond 
on  psychology  and  Tse  on  systems  theory,  provide  good  examples  of  thinking  not 
usually  brought  to  bear  in  OR/MS  research  and  have  been  reproduced  in  a  more 
extensive  report  on  the  workshop  (Little  1985). 


On  Forecasting  Research  Breakthroughs 

I  feel  more  than  a  little  uneasy  about  our  task  of  forecasting 
high  opportunity  research  topics.   The  most  important  research  is,  almost  by 
definition,  unpredictable.   For  example,  no  one  at  the  workshop  would  have 
dreamed  of  suggesting  an  improved  LP  algorithm  for  the  high  payoff  list,  but 
now  we  suddenly  have  one  and  the  payoff  looks  big. 

Nevertheless,  the  smug  observation  that  breakthroughs  cannot  be 
predicted,  often  made  by  researchers,  does  not  help  resource  allocation.   This 
is  true  both  at  the  personal  level  of  individuals  planning  their  own 
activities  and  at  the  institutional  level  for  organizations  like  NSF.   Whether 
we  are  good  at  it  or  not,  we  have  to  look  ahead. 

Another  concern  relates  to  the  casual  construction  of  wish  lists. 
The  popular  press  often  asks:   Why  doesn't  science  cure  the  common  cold  (or 
cancer),  do  away  with  poverty,  or  turn  garbage  into  gasoline.   The  reason  is 
that  good  scientific  research,  and  its  subsequent  practical  application, 
require  the  assembly  of  many  pieces.   These  include  at  least:   (1)  a  problem 
that  is  ready,  i.e.,  previous  knowledge  has  accumulated  sufficiently  that  the 
problem  can  be  defined  and  is  susceptible  to  solution,   (2)  a  methodology  or 
paradigm  that  is  available  or  can  reasonably  be  created  that  will  solve  the 
problem,   (3)  an  intellectual  or  economic  environment  that  makes  the  problem 
interesting  or  important  so  that  the  required  effort  will  be  expended,  and 
(4)  a  person  or  persons  who  appear  and  have  the  requisite  skills  to  perceive 
the  problem  and  conceive  of  a  way  to  solve  it.   The  required  skills  will  vary 
greatly  from  case  to  case.   One  situation  may  call  for  the  fresh  mind  of  a  23 
year  old  mathematician,  another  for  the  entrepeneurial  skills  of  a  senior 
researcher  running  a  large  organization. 

What  follows,  therefore,  is  not  so  much  a  wish- list  as  various 
thoughts  by  a  set  of  scientists  about  where  the  DMS  field  might  go,  could  go, 
or  should  go.   In  some  cases  the  researchers  are  self  critical  and  see  the 
necessity  of  expanding  their  current  paradigms;  in  others  they  see  technology 
opening  new  vistas;  and  in  still  others  a  clear  need  for  a  specific  pieces  of 
work.   In  almost  all  cases  they  believe  that  something  can  be  done,  i.e.,  the 
indicated  directions  are  feasible.   In  this  sense  I  suspect  the  suggestions 
are  conservative,  although  that  does  not  mean  that  results  will  necessarily 
come  easily.   Our  hope  is  that  the  articulation  here  will  help  start  some  set 
of  individuals  along  a  trail  of  ideas  that  will  lead  to  valuable  research. 
In  addition,  we  wish  to  sensitize  NSF  to  the  emerging  opportunities. 


HIGH  PAYOFF  OPPORTUNITIES  BY  DISCIPLINE 

This  section  summarizes  ideas  that  emerged  from  the  disciplinary 
presentations . 

Bass,  in  discussing  management  science,  highlights  opportunities 
surrounding  the  issues  of  individuals  making  choices,  drawing  illustrations 
from  his  own  field  of  marketing.   He  observes  that  marketing  measurements  are 
uncovering  fundamental  disagreements  with  the  assumptions  used  in  many 
economic  theories.   Understanding  what  is  going  on  at  the  individual  micro 
level  and  then  aggregating  it  to  the  macro  level  where  economists  normally 


work  offers  a  fruitful  avenue  for  research.   Also  related  to  choice, 
understanding  the  linkages  among  information  acquisition,  preference  for  an 
alternative,  intent  to  choose  it,  and  final  choice  require  deeper  study  by 
possibly  new,  empirical  methods.   Finally,  real  world  decision  makers  in 
managerial  roles  use  much  judgment  under  conditions  of  uncertainty  and  risk. 
There  is  a  need  for  what  might  be  called  "guessing  systems"  and  therefore 
deeper  fundamental  theories  of  how  people  do  and  should  employ  judgment  in 
actual  managerial  situations. 

Larson,  reporting  opportunities  in  operations  research,  returns  to 
the  writings  of  early  workers  in  the  field,  emphasizing  the  scientific  study 
of  operations  and  the  need  to  develop  identified  regularities  of  behavior  into 
natural  laws.   He  notes,  as  does  Bass,  the  need  to  go  from  micro  observations 
to  macro  laws.   In  speaking  of  congestion  processes,  such  as  the  distribution 
of  traffic  on  a  network,  he  observes  that  real  traffic  does  not  always  behave 
according  to  the  predictions  of  the  equilibrium  models  devised  to  describe  it. 
Similarly,  most  queuing  systems  in  practice  violate  many  of  the  standard 
assumptions  of  the  existing  theory.   The  discrepancy  between  theory  and  actual 
observation  provides  opportunities  for  new  theories  and  new  practice. 

A  neglected  area  is  the  development  of  tools  for  robust  analysis. 
Larson  cites  a  low-tech  solution  for  an  appplication,  a  "meals  on  wheels"  food 
delivery  service.   Researchers  devised  a  pencil-and-paper  scheduling  procedure 
that  featured  simplicity  and  robustness  rather  than  sophistication  and 
optimality.   This  provided  excellent  operating  results  when  put  to  the  stress 
test  of  field  use  by  non- technical  people.   Larson  expresses  a  desire  for  less 
research  that  refines  existing  models  and  more  that  invents  new  models  for  new 
operational  contexts,  pointing  out  that  the  historical  strength  of  OR  has  been 
the  ability  to  give  structure  to  new  problems.   He  conjectures  that  in  recent 
years,  OR/MS  has  spun  off  more  in  the  way  of  subfields  than  it  has  acquired  in 
the  way  of  new  problems  to  work  on.   Finally  he  sees  great  payoff  from  the  use 
of  super-micro  computers  and  graphics  as  implementation  vehicles  for  OR/MS 
models. 

Hiimmond's  provocative  paper  lays  out  high  payoff  opportunities  in 
pyschology  within  DMS .   He  first  describes  potential  contributions  to  DMS  from 
the  pyschological  literature.   Work  carried  out  in  the  1970 's  on  how  people 
actually  make  judgments  and  decisions  has  essentially  demolished  the  validity 
of  the  expected  utility  model  for  describing  people's  decision  making 
behavior.   Many  ingenious  laboratory  experiments  have  demonstrated  that,  under 
a  variety  of  conditions,  people  will  consistently  make  different  decisions 
from  those  specified  by  the  standard  rational  models  of  decision  analysis. 
This  discovery  provides  a  rich  set  of  research  opportunities.   For  example, 
what  are  the  costs  of  non-rational  behavior  in  various  contexts  and  what  are 
potential  remedies  for  such  behavior? 

Another  branch  of  pyschological  literature  compares  the  judgments  of 
a  person  with  the  outputs  of  an  empirical  formula.   It  is  found  over  many 
cases  that  simple  linear  models  predict  most  task  outcomes  better  than  expert 
judgment.   Further  it  has  been  found  that  these  simple  additive  or  weighted 
sum  models  predict  judgments  quite  accurately.   The  main  implication  of  this 
research  is  that,  in  many  cases,  linear  models  rather  than  a  person  should  be 
given  the  information  necssary  for  prediction,  even  if  the  person  is  an 
expert.   This,  too,  opens  up  a  variety  of  DMS  relevant  research,  such  as 
detetermining  differential  costs  and  benefits  of  the  use  of  these  models  in 


operational  situations. 

In  contrast,  psychological  researchers  from  the  field  of  problem- 
solving  have  found  that  people,  especially  experts,  store  large  amounts  of 
information  and  use  them  effectively,  often  in  the  form  of  short  cuts.   Notice 
that  this  research  stream  and  the  previous  one  are  strangely  contradictory 
about  experts.   The  judgement  and  decision-making  researchers  usually  find 
that  experts  fail  and  should  be  replaced  by  a  simple  linear  processing  of 
inputs,  vhereas  the  problem- solving  researchers  find  that  experts  are  superior 
problem- solvers .   What  are  the  differences  in  settings  and  tasks  that  give 
rise  to  these  contradictory  outcomes?   What  is  correct?   As  Hammond  says,  it 
is  hard  to  think  of  a  more  important  problem  for  Decision  and  Management 
Science . 

The  DMS  charter  calls  for  generality  of  results,  but  Hammond 
observes  that  in  all  three  of  the  research  literatures  just  cited 
psychologists  have  found  their  results  to  be  highly  task  dependent. 
Generality  is  disappointingly  lacking.   This  means  that  DMS  researchers  must 
be  cautious  in  building  on  the  work  of  the  pyschologists ,  but  therein  also 
lies  a  major  opportunity.   Hammond  feels  that  DMS  can  contribute  much  to 
pyschology  as  well  as  itself  by  requiring  generalizable  results  in  operational 
settings.   Traditional  psychological  research  is  laboratory-bound,  often  with 
inexpensive  research  projects  and  students  as  subjects.   He  cites  examples  of 
advances  in  psychological  research  that  he  feels  cannot  be  made  unless  the 
work  satisfies  the  DMS  policy  guidelines.   These  include  understanding 
decision  making  over  time,  understanding  expert  judgment,  and  understanding 
group  decision  making  among  people  in  real  organizations. 

Einhom,  in  his  discussion  of  opportunities  for  DMS  research  in 
behavioral  decision  making,  zeros  in  on  a  major  issue  raised  by  Hammond, 
namely,  generalizability.   Work  in  the  pyschological  literature  always  seems 
to  end  up  qualifying  the  results  with  "it  depends."   Behavior  is  found  to  be 
highly  contingent  on  the  environment.   The  environment  is  infinitely  variable 
and  complex.   As  a  result  there  is  a  tendency  to  despair  about  obtaining 
general  principles  of  behavior.   He  asks  whether  we  are  doomed  to  an 
enumeration  of  highly  specific  conditions. 

His  answer  is  that  we  need  models,  just  as  stated  in  the  goals  of 
DMS.   He  goes  on  to  illustrate  the  idea  by  analyzing  a  particular  situation. 
In  communications  research,  subjects  have  been  found  to  be  persuaded  most  by 
information  presented  to  them  most  recently  -  in  some  cases.   In  others,  the 
information  presented  first  has  the  greatest  effect.   In  still  other  cases, 
the  order  of  presentation  does  not  matter.   Einhom  suggests  that  the 
resolution  of  these  contradictory  results  will  lie  in  a  model  that  explicitly 
accounts  for  differences  in  the  conditions.   For  example,  the  model  should 
contain  at  least  two  counteracting  forces,  which  are  then  linked  to 
explanatory  variables  describing  the  environment.   He  goes  on  to  hypothesize 
such  a  model  and  show  how  it  could  work  to  explain  the  observed  behavior.   His 
point  is  clear:   traditional  psychological  research  has  focussed  on  varying 
the  conditions  and  observing  the  effects  and  not,  as  required  by  the  DMS 
mission  statement,  on  building  models  that  would  explain  behavior  across  many 
experiments.   Therein  lie  the  research  opportunities,  both  for  psychology  and 
Decision  and  Management  Science. 

Tse  explores  opportunities  in  systems  theory  for  DMS  objectives. 


Although  systems  theory  is  closer  to  standard  OR/MS  in  heritage  than  the 
pyschology,  Tse  has  some  rather  different  perspectives.   First  he  lays  out  a 
multi-stage  view  of  decision  making.    In  it  he  differentiates  between  a 
"generation"  phase,  which  consists  of  problem  recognition,  issue 
identification,  option  generation,  and  preliminary  option  screening,  and  a 
"choice"  phase,  which  contains  selection  and  implementation.   Although 
research  opportunities  exist  at  each  stage,  Tse  points  particularly  to  the 
study  of  intelligent  activity  in  decision  making  as  an  area  of  promise  opened 
up  by  developments  in  artificial  intelligence  and  expert  systems.   Further 
special  opportunities  arise  because  of  recent  developments  in  man-computer 
interfaces.   Moreover,  he  specifically  calls  for  a  new  framework  to  handle  the 
generation  and  choice  paradigm.   He  sees  an  opportunity  for  finding  a 
representational  framework  that  handles  uncertainty  and  copes  with  semantic 
ambiguity.   One  potential  approach  is  the  possibility  theory  of  Zadeh. 

Tse  also  sees  the  need  for  the  development  of  methodologies  to 
enhance  the  dynamic  iteration  between  generation  and  choice.   Such  research  is 
likely  to  require  the  integration  of  expert  systems,  user  interface 
technology,  systems  analysis  and  human  interaction.   His  vision  is  that,  with 
the  help  of  interactive  computing,  a  simulation  environment  can  be  built  in 
which  the  system  equations  are  parameterized.   Expert  systems  would  contain 
rules  for  selecting  equation  structures  to  be  simulated.   Such  an  interactive 
simulation  and  expert  environment  would  allow  the  user  to  change  rules,  model, 
and  parameters  interactively  during  system  simulation.   With  the  computer 
taking  care  of  the  analytical  manipulations,  the  user  can  experiment  with  the 
implications  of  new  options.   This  would  permit  the  uncovering  of  new  issues 
that  in  turn  would  redirect  the  search  for  alternative  options  in  a  dynamic 
iteration  between  the  generation  and  choice  phases  of  problem  solving. 


CROSS-DISCIPLINARY  OPPORTUNITIES 


A  major  goal  of  the  workshop  was  to  bring  together  individuals  with 
quite  different  backgrounds  to  identify  research  topics  in  DMS .   In  terms  of 
process,  the  group  as  a  whole  developed  a  list  of  topics.   These  were  then 
coalesced  under  major  headings,  after  which  the  conference  split  into  mixed- 
discipline  working  groups,  each  of  which  sought  to  articulate  the  research 
opportunities  under  one  assigned  heading. 

After  simplifying  some  of  the  titles  and  rearranging  them  slightly, 
I  have  organized  the  content  as  follows: 

1.  Operational  processes 

a.  Phenomenological  processes 

b.  Organizational  and  managerial  processes 

2.  Choice  theory 

a.  Individual  choice,  values,  judgment,  and  risk  behavior 

b.  Group  choice  and  decision  making 


3.   Decision  support  and  expert  systems 

A.   Complexity 

Within  each  heading  I  shall  try  to  capture  key  ideas  that  percolated  up  during 
the  conference.   In  doing  this  I  am  indebted  to  the  people  who  summarized  the 
discussions  within  each  working  group.   In  several  cases  their  notes  are 
reproduced  almost  verbatim.   However,  responsibility  for  editorial  injustice 
is  mine. 

1.  Operational  Processes 

a.  Phenomenological  processes. 

The  word  phenomenological  is  not  only  a  mouthful  but  needs 
elaboration  and  example.   First,  we  divide  such  processes  into  natural  and 
designed.   Natural  processes  are  those  that  occur  without  an  organized, 
conscious  effort  to  make  each  individual  event  happen:  for  example,  traffic  on 
a  road  network,  customers  arriving  at  a  store,  or  viewers  switching  TV 
channels.   Designed  processes  have  a  structure  deliberately  put  together  by  an 
organization:  for  example,  a  production  line,  airplane  landings,  or  office 
administrative  procedures. 

Within  natural  processes  three  kinds  can  be  identified  that  warrant 
modeling  and  empirical  observation  in  multiple  contexts.   The  first, 
"diffusion  processes,"  includes  propagation  of  information,  technology 
transfer,  diffusion  of  innovation,  spread  of  epidemics,  etc.   The  second, 
"horaeostatic  processes,"  refers  to  systems  that  tend  to  retain  stable 
characteristics  even  in  a  changing  environment.   The  third  group, 
"evolutionary  processes,"  encompasses  growth  and  change  in  many  entities, 
e.g.,  cities,  corporations,  and  communications  networks. 

Designed  processes  of  particular  interest  are  of  two  kinds:   "flow 
processes"  and  "response  processes."   The  "flow  processes"  include  a  wide 
variety  of  situations  where  tangible  items  (such  as  partially  completed 
products  on  an  assembly  line)  or  intangible  entities  (such  as  information) 
flow  in  some  purposive  way  along  paths  that  may  be  individually  charted  or 
collectively  established  but  proceed  with  the  intention  of  achieving  some 
outcome.   These  processes  may  also  suffer  congestion.   "Response  processes" 
are  exemplified  by  a  wide  variety  of  emergency  service  systems  (e.g.,  fire, 
ambulance,  police). 

A  research  goal  is  to  develop  classes  of  models  that  capture  a  rich 
array  of  contexts  in  one  or  more  of  the  categories  mentioned.   General 
approaches  will  be  useful,  some  perhaps  micro  at  the  individual  event  level, 
others  macro  describing  aggregate  behavior.   Ideal  would  be  general  models 
with  parameters  that  can  be  measured  in  specific  contexts  to  particularize  the 
application.   The  task  may  not  be  easy  but  there  are  important  commonalities 
between  quite  different  phenomena  and  the  discovery  of  general  models  that 
permit  the  determination  of  properties  of  various  operations  will  be  of 
significant  value. 


b.    Organizational  and  managerial  processes. 

Three  areas  provide  major  opportunities:   (1)   organizational  design 
for  decision  making,  (2)  organizational  support  systems,  and  (3)  strategic 
management . 

In  organizational  desip^^  for  decision  making  the  scientific  and 
practical  goals  are  clear:   We  would  like  to  connect  the  decision  making  tasks 
to  the  design  parameters  of  the  organization,  then  relate  the  design  to  the 
eventual  outcomes  of  the  decision  making,  and  finally  relate  those  outcomes  to 
relevant  measures  of  mission  performance.   In  format  this  is  classical  OR/MS 
and  systems  theory,  but,  as  of  today,  we  almost  completely  lack  the 
quantitative  linkages  required.   Not  only  are  the  relevant  descriptors  and 
parameters  of  organizational  design  large  in  number,  but  the  relevant 
processes  include  such  poorly  modeled  issues  as  managerial  control  and 
distributed  decision  making. 

The  members  of  the  mixed-discipline  group  see  organizational  design 
as  enormously  important  and,  of  course,  difficult,  but  they  are  very  clear  and 
unanimous  on  the  need  to  approach  it  with  formal  quantitative  models.   These 
are  likely  to  require  the  development  of  new  language  and  terminolgy. 
Unquestionably,  theory-driven  field  experiments  will  be  needed.   These  may  be 
naturally  occurring  or  specifically  designed.   Visualized  is  a  multiple 
step  research  endeavor  that  starts  with  the  qualitative  knowledge  already 
built  up  in  organizational  science,  proceeds  by  devising  mathematical         ' 
theories,  continues  with  the  development  of  methods  for  making  field 
measurements,  and  culminates  with  tests  of  the  theories  in  the  field  with  real 
organizations.   Although  this  does  not  preclude  laboratory  tests,  field 
testing  should  be  the  attained  goal. 

If  our  theories  and  measurements  become  good  enough,  we  shall  be 
able  to  consider  computer-aided  design  of  organizational  systems. 

A  second  major  theme  is  organizational  support  systems. 
Organizations  exist  and  function  through  a  variety  of  information  and 
communication  processes.   These  have  received  considerable  research  although 
formal  models  of  them  are  few.   We  are  now  in  an  era  of  rapid  change  in 
information  processing  and  communications  technology.   This  situation  offers  a 
remarkable  opportunity  for  studying  these  processes  because  the  changes  are 
creating  innumerable  natural  experiments.   Part  of  the  picture  and  a  vital 
topic  in  its  own  right  is  the  effect  of  specific  kinds  of  computer  and 
communications  hardware  and  software  on  managerial  processes. 

A  third  major  opportunity  for  applying  the  DMS  guidelines  lies  in 
strategic  management.   This  field  underwent  great  growth  in  the  1970' s  both  as 
a  consulting  practice  and  as  an  internal  corporate  activity.   Much  of  the  work 
has  rested  on  a  few  simple,  but  quite  powerful  conceptual  frameworks.   Since 
the  world  we  live  in  seems  to  be  increasing  its  rate  of  change  and  certainly 
is  increasing  in  inter-connectedness  and  interdependence,  the  role  of 
strategic  management  is  secure.   An  organization  must  sense  its  external 
environment  and  internal  state  and  then  develop  and  execute  short  and  long 
range  plans.   Needed  are  models  to  describe  the  environment  and  relate 
alternative  strategic  thrusts  to  outcomes  and  performance.   Since  most 
external  environments  contain  competitors,  our  models  must  provide  for 
competitive  behavior  and  response.   Developments  in  the  field  of  differential 


games  may  be  useful.   In  any  case  we  need  new  models  of  markets  and 
organizations  to  support  the  overall  planning  process. 

Only  through  interdisciplinary  research  will  progress  be  made  on 
these  topics.   OR/MS  and  systems  theory  can  contribute  mathematical  modeling 
and  organizational  science  can  offer  measurement  skills  and  qualitative 
theories.   It  is  often  difficult  to  find  people  from  different  disciplines  in 
the  same  organization,  e.g.,  the  same  university  department,  but  there  may  be 
ways  to  reduce  the  barriers.   For  example,  NSF  can  encourage  interdisciplinary 
conferences  and  interdisciplinary  proposals. 

Another  obstacle  is  the  very  size  and  scope  of  the  research  tasks 
described,  to  say  nothing  of  the  difficulties  of  mounting  research  efforts  on 
real  organizations.   An  answer  to  this  is  to  conduct  research  in  phases.   The 
first  phase  is  to  construct  theory  and  models  on  the  basis  of  current 
knowledge.   The  second  is  to  devise  measurement  systems  and  methods  of 
instrumentation  within  organizations.   The  third  involves  the  actual  field 
data  collection.   Finally,  the  fourth  phase  consists  of  data  analysis  and 
testing  of  the  theories.   Notice  that  each  stage  has  stand  alone  interest  and 
should  produce  important  publishable  research.   However,  the  work  should  be 
shaped  and  driven  by  the  larger  research  goals  of  the  overall  project. 

Some  of  the  supporting  disciplines  for  addressing  these  major 
questions  are  organizational  science,  non-zero  sum  differential  games,  the 
cognitive  side  of  expert  systems,  theories  of  distributed  information  systems, 
modeling  concepts  from  large  scale  systems,  and  multivariate  statistical 
methods,  such  as  structural  equations,  that  support  model  calibration. 


2.    Choice  theory 

a.   Individual  choice,  values,  judgment,  and  risk  behavior 
Four  important  research  areas  with  high  promise  are: 

(1)  Normative  theory  and  human  behavior.   Many  empirical  studies 
have  shown  real  decision  making  to  be  different  from  that  prescribed  by 
traditional  rational  models.   While  this  can  be  interpreted  as  demonstrating 
irrationality,  it  can  also  be  seen  as  challenging  the  traditional  conception 
of  irrationality,  calling  perhaps  for  new  normative  theories.   Anticpated 
regret,  costs  associated  with  thinking,  and  alternative  axiom  systems  for 
utility  have  recently  been  proposed  to  account  for  the  observed  anomalous 
behavior.   Work  that  proposes  and  tests  alternative  or  broader  conceptions  of 
rationality  deserves  high  priority. 

(2)  Behavioral  bases  of  decision/risk  analysis.   Formal  methods  for 
aiding  decisions,  such  as  decision  analysis  and  risk  analysis,  depend 
critically  on  inputs  from  individuals.   Judgments  of  probability  have  been 
extensively  studied,  although  unanswered  questions  remain.   Several  other 
processes  are  far  less  studied  and  need  attention.   These  include  problem 
structuring  and  option  generation,  the  elicitation  and  modeling  of 
preferences,  and  the  development  of  methods  to  extract  information  from 
experts. 


A  great  deal  of  research  has  documented  the  existence  of 
systematic  biases  in  judgments  under  uncertainty.   Research  is  now  needed  to 
find  ways  to  reduce  the  biases.   We  need  better  understanding  of  risk 
preferences.   Multiple  criteria  models  should  be  improved,  not  only  for  simple 
decisions,  but  also  for  individuals  faced  with  multiple  decisions. 

(3)  Descriptive  choice  theory/customer  behavior.   As  with  decision 
analysis,  we  need  research  aimed  at  illuminating  what  goes  on  in  early  stages 
of  choice.   Topics  such  as  framing  and  agenda  effects  and  option  generation 
are  key  here.   Additional  work  should  elaborate  the  specific  cognitive 
processes  that  underlie  choice  behavior.   The  relationship  between  preference, 
intention,  and  choice  is  not  well  understood. 

Also  of  importance  are  the  possible  effects  of  the  new 
information  and  communication  technologies  on  individuals'  decision-making 
behavior.   Unanswered  questions  remain  about  how  decision  makers  cope  with 
uncertainty.   Do  they  view  their  problems  as  multi-outcome  lotteries,  or  do 
they  employ  other  models? 

(4)  Decision-making  in  crises.   Crisis  situations  pose  an  extreme 
challenge  to  the  abilities  of  decision  researchers.   Such  situtations,  whether 
in  military  or  non-military  settings  (accidents,  swift  changes  in  the  business 
environment,  etc.)  are  characterized  by  high  stakes,  severe  time  pressures, 
fear,  and  other  elements  of  stress.   Attempts  to  cope  with  potential  crises 
usually  take  the  form  of  contingency  planning  and  simulation.   Such  activities 
occur  in  hypothetical  settings,  far  distant  in  time  from  the  actual  crisis. 

We  need  research  to  help  us  understand  the  effects  of  stress,  hypotheticality , 
temporal  distance,  etc.,  on  decision  making.   For  example,  will  utility 
functions  for  gains  and  losses  change  systematically  as  one  gets  closer  in 
time  to  experiencing  a  crisis?  How  can  realistic  simulations  be  created  for 
training  purposes? 

Besides  the  four  primary  research  topics  just  discussed,  two  cross- 
cutting  issues  are  germane  to  all  of  them:  temporal  factors  and  the 
methodological  question  of  laboratory  vs.  field  studies. 

Temporal  interactions  add  complexity  to  all  the  research  questions 
posed  above  and  are  of  obvious  theoretical  and  practical  importance.   The  role 
of  time  in  crisis  decision  making  has  been  noted.   In  addition  there  is  the 
need  to  study  the  interplay  between  risk  attitude  and  the  decision  maker's 
planning  horizon  (the  modeling  of  risk  preferences  in  multiple  time  periods) . 
An  important  problem  in  consumer  behavior  is  understanding  the  pattern  of  an 
individual's  decisions  over  time.   We  need  better  methods  to  detect  change  and 
to  model  the  stochastic  processes  involved. 

Finally,  both  laboratory  and  field  studies  will  play  a  role  in 
addressing  the  topics  described.  Particularly  promising  is  the  use  of  a 
combined  laboratory- field  approach  sequentially  within  the  same  project. 

b.    Group  choice  and  decision  making. 

Under  this  title  six  principal  research  topics  emerge: 
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(1)  Structuring  problems  with  groups.   Needed  research  includes 
work  on  option  generation,  value  structuring,  and  the  issues  raised  by 
multiple  criteria. 

(2)  Organizational  information.   Relevant  are  the  generation  of 
information,  its  representation,  and  how  it  affects  interlocking  and 
interdependent  decision  making. 

(3)  Conflict  resolution.   This  is  an  important  area  which  has 
received  much  attention  in  various  forms  but  continues  to  elude  a  thorough 
scientific  understanding.   Theories  and  measurements  for  bargaining  and 
negotiation  are  required.   Related  are  more  general  considerations  of  multi- 
party decision  making.   Subproblems  include  the  study  of  fair  allocation  and 
such  phenomena  as  suppression  of  issues  during  negotiation. 

(4)  Values  and  judgments.   As  mentioned  elsewhere  we  need  to  model 
the  experts.   Research  is  also  required  on  value  structures,  preference,  and 
risk  assessment  in  groups.   Beyond  that  comes  the  question  of  how  these 
structures  and  assessments  evolve  over  time.   A  neglected  issue  for  the 
researcher  is  how  the  model  builder's  values  may  differ  from  those  of  the 
model  user. 

(5)  Organizational  implementation.    We  wish  to  understand  not  only 
how  a  group  makes  its  choices  and  decisions  but  how  it  carries  them  out,  for 
this  undoubtedly  feeds  back  into  how  the  decision  is  made.   Organizational 
structure  is  a  variable  here  and  questions  arise  over  timing  and  various 
organizational  interdependencies . 

(6)  Decision  making  over  time.   A  recurring  theme  is  that,  as 
researchers,  we  start  with  circumscribed,  usually  static  situations  in  our 
quest  for  understanding,  but  the  world  is  dominated  and  driven  by  dynamic 
processes.   These  clearly  feed  back  into  the  decision  process  and  so  deserve 
study  and  consideration  for  themselves. 

Progress  on  these  difficult  topics  requires  interdisciplinary 
research,  which  is  well-known  to  have  many  barriers.   To  help  overcome  these 
obstacles:   (1)  link  research  to  real  world  problems;  (2)  follow  the 
guidelines  of  DMS  advisory  panel;  (3)  develop  postdoctoral  programs  for 
interdisciplinary  training  (stipends  to  be  awarded  for  work  outside  the 
researcher's  usual  specialty);  (4)  focus  grants  on  multi-disciplinary 
projects;  (5)  sponsor  multi-disciplinary  conferences  on  topics  requiring  such 
knowledge  and  skills. 

3.    Decision  Support  and  Expert  Systems 

Eight  topics  have  been  collected  under  the  heading  of  decision 
support  and  expert  systems:   (1)  knowledge  representation,  (2)  decision 
analysis  with  expert  systems,  (3)  person/system  interaction,  (4)  impact  of 
real  time  information  on  decision  making,  (5)  information  requirements 
determination,  (6)  information  systems  impact  on  organizational  structure,  (7) 
generation  of  heuristics,  and  (8)  limits  of  machine  generated  decision  making. 

To  animate  and  focus  the  issues,  subsets  of  these  topics  have  been 
worked  into  suggested  projects  for  NSF/DMS  funding.   Of  course,  given  the 
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limited  time  at  the  workshop,  ntomerous  details  remain  to  be  developed. 
Nevertheless,  the  skeletons  of  three  quite  interesting  projects  emerge: 

(1)   Interplay  between  expert  systems,  decision  analysis, 
models/algorithms,  and  humans 

Decision  makers  face  a  spectrum  of  decision  complexity,  ranging  from 
frequent,  rather  routine  decisions  in  statistically  regular  environments  to 
infrequent  decisions  with  information  profiles  that  may  be  unprecedented. 
Moreover,  paralleling  this  spectrum  are  utilities  or  disutilities  associated 
with  outcomes,  which  may  range  from  relatively  small  in  absolute  value  for 
routine  decisions  to  very  high  (perhaps  with  large  life-and-death 
consequences)  for  non-routine,  very  rare  decisions. 

The  research  question  focuses  on  the  allocation  of  this  spectrum  to 
various  techniques  that  are  now  available  for  decision  aiding.   A  hypothesis 
is  that  expert  systems,  somehow  married  with  models  and  algorithms,  would  be 
excellent,  perhaps  superlative  decision  aiders  and/or  decision  makers  for  the 
statistically  regular,  high  frequency  decisions;  that  formalized  decision 
analytic  techniques  would  be  appropriate  for  more  difficult,  less  frequent 
decisions  with  high  possible  disutility;  and  that  perhaps  no  formal  techniques 
could  be  relied  on  for  points  near  the  end  of  the  spectrum  of  statistical 
irregularity  and  highest  potential  disutility.   Throughout  the  spectrum,  even 
when  it  is  envisioned  that  formal  techniques  may  be  superior  to  human 
judgment,  a  question  remains  as  to  the  suitability  for  "human  override"  of  the 
formal  technique  and  the  appropriate  methods  for  evaluating  the  desirability 
and  correctness  of  a  selected  human  override  technique. 

The  spectrum  of  decisions  and  decision  complexities  can  be  brought 
out  by  the  study  of  real  systems,  such  as  emergency  services  (police,  fire, 
and  ambulance),  and  diagnostic  operations  (such  as  fault  diagnosis,  medical 
diagnosis,  and  analysis  of  medical  tests).   Response  to  a  suspected  nuclear 
attack  would  be  an  obvious  candidate  for  examining  the  most  difficult  end  of  ' 
the  spectrum. 

Research  questions  abound  under  this  umbrella  of  topics.   For  one, 
there  is  the  question  of  the  appropriate  marriages  between  expert  systems 
and  traditional  operations  research  models  and  algorithms.   Expert  systems 
often  make  use  of  detailed  lists  of  conditions  giving  rise  to  a  particular 
action;   OR  models  and  algorithms  tend  to  rely  on  statistically  regular 
physical  operations  and  models  of  systems,  and  derive  their  normative 
implications  from  analysis  of  the  model's  mathematical  properties.   In  some 
senses,  expert  systems  parallel  the  thinking  of  the  clinician,  who 
conceptualizes  his  activities  on  a  case-by-case  basis;  whereas  OR  models 
parallel  the  actions  of  a  statistician,  who  relies  on  regularities  derived 
from  large  populations . 

Attributes  that  seem  associated  with  expert  systems  and/or 
traditional  OR  models  and  algorithms  are:   structured,  high  in  certainty, 
clear-cut,  algorithmic,  deterministic,  repetitive,  and  systematic.   Attributes 
toward  the  other  end  of  the  spectrum  of  decision  making  that  seem  to  require 
extensive  judgment  and  perhaps  formal  use  of  decision  analysis  are: 
unstructured,  uncertain,  ambiguous,  judgmental,  probabilistic,  breaking  of 
historical  precedent,  value  laden,  and  life  consequential. 
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The  likely  end- implementation  of  this  line  of  research  would  be  an 
allocation  of  the  decision  difficulty  spectrum  to  machines  and  humans.   A 
most  critical  type  of  situation  would  be  one  in  which,  say,  99.9  percent  of 
the  time,  the  machine  provides  the  correct  decision,  but  when  an  error  is 
made,  the  results  may  be  catastrophic.   Obvious  examples  are:  decisions 
regarding  nuclear  retaliation  to  presumed  nuclear  attack  where  the  timeframe 
for  decision  making  is  6-8  minutes,  emergency  maneuvers  for  aircraft 
attempting  to  avoid  mid-air  collisions,  decisions  on  real-time  control  of 
nuclear  reactors  facing  emergency  conditions,  etc. 

(2)  Person/machine  as  a  decision  making  unit:  effective 
organization,  presentation,  and  exchange  of  information. 

The  overall  objective  of  this  project  is  to  study  alternative  and 
improved  mechanisms  for  information  exchange  from  the  machine  (i.e., 
computer)  to  the  user  and  from  the  user  back  to  the  machine.   The  work  is 
motivated  in  part  by  the  rapidly  advancing  technologies  in  computer  graphics, 
which  are  making  obsolete  the  simple  alpha-numeric  computer  display  of 
information.   Today  a  rich  assortment  of  window-oriented  graphic  displays, 
some  using  color,  can  provide  a  much  richer  environment  for  providing  the  user 
with  information.   In  the  reverse  direction,  the  user  can  convey  information 
and  instructions  to  the  computer  in  numerous  ways  other  than  the  perhaps -to- 
become  -outmoded  keyboard;   these  include  mouse,  lightpen,  joystick,  touch 
sensitive  screens,  voice  activation,  and  others. 

To  carry  out  the  project  requires  one  or  more  simulation 
laboratories  in  which  to  test  hypotheses  about  information  transfer  between 
people  and  computers.   The  laboratories  need  state-of-the-art  equipment  in 
computers,  computer  graphics,  etc.   Research  questions  include  alternative 
ways  to  present  data  (including  use  of  color  and  graphics);   presentation  of 
only  decision-relevant  information  (to  avoid  information  overload);  uses  of 
sound,  touch,  and  other  senses  for  information  conveyance;  ways  to  design  the 
person-machine  unit  so  as  to  optimize  the  balance  between  the  on-line 
computational  power  of  the  computer  and  the  judgmental  decision  power  of  the 
human. 

(3)  Information  systems  impact  on  organizational  (and  decision 
making)  structure 

The  key  conjecture  behind  this  project  is  that  the  revolution  in 
networked  local  computation  that  we  are  currently  experiencing  will  have 
marked  effects  on  organizational  sturcture  and  behavior.   The  organization 
as  an  organism  may  experience  a  dramatically  swift  Darwinian  change 
(mutation?)  as  a  consequence  of  a  redesign  of  its  "nervous  system,"  (i.e., 
its  network  for  information  transmittal  and  storage) .   Since  knowledge  is 
power  within  organizations.  It  is  possible  that  the  traditional  hierarchical 
organization  will  have  difficulty  in  maintaining  its  hierarchical  structure 
in  an  era  of  decentralized  computation  and  information  transfer. 

One  way  to  conduct  this  project  is  to  study  current  organizations' 
responses  to  the  introduction  of  networked  information  systems  as  a  set  of 
naturally  occuring  experiments. 

Under  the  umbrella  of  the  project,  numerous  related  research 
questions  arise:   Are  currently  implemented  and  proposed  MIS  systems  more 
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democratic  or  authoritarian  in  terms  of  organizational  structure?   Are  new 
networked  systems  yielding  a  higher  degree  of  decentralization  than  we  have 
seen  in  the  mainframe  era?   How  are  decentralized  networked  MIS  systems 
affecting  individuals,  their  incentives,  reward  structures,  and  the  quality  of 
their  working  life? 


4.    Complexity 

Complexity  affects  DMS  research  in  three  conceptually  distinct 
forms.   These  can  be  characterized  as  prescriptive,  descriptive  and 
communicative  complexity. 

(1)  Prescriptive  complexity. 

DMS  researchers  whose  goal  is  to  help  decision  makers  make  better 
decisions  must  cope  with  complexity  in  the  decision  setting.   For  example, 
in  the  design  of  complex  systems  there  are  many  separate  steps  or  processes 
each  of  which  feeds  back  into  and  affects  all  the  others.   We  call  this 
structural  complexity.   Similarly,  in  problem  domains  that,  in  principle, 
can  be  modeled  statistically,  difficulties  frequently  arise  when  the 
processes  under  study  are  noisy  or  mutually  dependent.   This  is  statistical 
complexity. 

The  time  is  ripe  for  DMS  researchers  to  tackle  problems  such  as 
these.   In  general,  enough  is  now  known  and  methods  are  sufficiently  well 
developed  for  research  on  these  topics  to  be  pursued  fruitfully.   There  are  at 
least  two  useful  directions.   One  is  for  "outside  persons"  to  observe  and 
study  interactions  in  complex  settings.   Thus,  a  behavioral  scientist  might 
observe  users  and  system  designers  interacting  in  the  development  of  a  complex 
system,  with  intervention  and  experimental  manipulation  employed  as 
appropriate.   A  second,  complementary  activity  would  be  workshops  focused  on 
problems  which  are  reasonably  likely  to  be  soluble  at  present  but  which,  for 
whatever  reasons,  have  received  insufficient  attention  from  relevant 
researchers.   One  such  problem  is  the  development  of  techniques  for  modeling 
complex  statistical  dependencies  in  statistical  inference. 

(2)  Descriptive  complexity. 

Many  DMS  researchers  are  primarily  interested  in  the  description  of 
the  decision  making  processes  of  groups  or  individuals.   Although  current 
research  on  these  topics  has  illuminated  certain  features  (e.g.,  heuristic 
methods  used  by  naive  individuals,  dynamic  processes  in  groups),  much  needs  to 
be  done  to  understand  the  functioning  of  these  processes  in  the  complicated 
environments  in  which  they  naturally  occur.   In  particular,  strong  theoretical 
work,  perhaps  involving  simulation,  is  needed  to  define  not  only  the 
characteristics  of  the  processes  (heuristics)  themselves,  but  also  the 
functional  characteristics  of  the  environments  that  control  outcome- 
ef f ectiveness . 

Studies  of  this  kind  are  feasible  now,  but  generally  would 
require,  in  addition  to  ordinary  research  investigator  time,  research 
assistants,  and  access  to  powerful  computing  equipment  via  medium-sized 
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computers  or  networks  linked  to  supercomputers.   NSF  (in  general,  and  also  in 
conjunction  with  specific  DMS  programs)  could  clearly  be  of  major  importance 
in  making  these  facilities  possible. 

(3)   Communicative  complexity. 

DMS  is  a  multi-disciplinary  area.   The  need  is  for  real 
hybridization  not  Just  a  collection  of  disciplines.   Interdisciplinary 
research,  however,  requires  participants  who  have  reasonable  familiarity  with 
one  another's  fields,  including  not  only  matters  of  the  language  in  which 
problems  are  discussed,  but  also  the  goals  and  "world  views"   of  the  various 
practitioners.   Such  familiarity  is  not  easily  acquired  across  disciplinary 
boundaries.   Encouragement  notwithstanding,  it  is  difficult  for  researchers 
from  non-overlapping  specialities  to  come  up  with  interdisciplinary  proposals 
without  some  preliminary  interaction. 

Two  ways  to  foster  such  speculative  interaction  are  (1)  to  develop 
support  (i.e.,  for  released  time,  administrative  costs,  etc.)  for  potentially 
interdisciplinary  groups  at  campuses  where  the  right  mix  of  people  already 
exists,  but  where  no  mechanisms  exist  to  make  it  easy  for  such  people  to 
become  acquainted  and  (2)  to  fund  postdoctoral  appointments  or  extended  vists 
of  DMS  researchers  from  one  field  to  laboratories  where  research  from  some 
other  DMS  field  is  being  done.   In  both  cases  the  intent  is  to  make  access  to 
people  in  other  fields  easier  than  is  currently  the  case. 

Meaningful  research  along  many  of  the  lines  reported  in  the  workshop 
will  not  be  quick.   Much  of  it  requires  multiple  stages,  development  of 
relationships  with  external  organizations  as  sites,  and  specialized  equipment. 
A  3-5  year  time  frame  is  realistic  in  many  cases.   Therefore,  it  is 
recommended  that,  for  research  involving  operating  systems  in  the  real  world, 
long-term  support  be  sought  that  recognizes  the  special  nature  of  the 
relationships  required  between  researchers  and  the  organizations  being 
studied. 

Small  workshops  focused  on  priority  cross-disciplinary  issues  have 
demonstrated  their  effectiveness.   It  is  recommended  that  a  DMS  workshop 
similar  to  this  one  be  held  on  an  annual  or  other  periodic  basis. 
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APPENDIX  I 
Workshop  Participants 


Participants  in  the  Dallas  workshop  were: 

Frank  Bass,  University  of  Texas  at  Dallas; 

Alfred  Blumstein,  Carnegie-Mellon  University; 

Emilio  Casetti,  Ohio  State  University; 

Michael  Cohen,  University  of  Michigan; 

Rudolph  Drenick,  Polytechnic  Institute  of  New  York; 

Hillel  Einhorn,  University  of  Chicago; 

Peter  Farquhar,  Carnegie-Mellon  University; 

Gregory  Fischer,  Carnegie-Mellon  University; 

Kenneth  Hammond,  University  of  Colorado; 

George  Ruber,  University  of  Texas  at  Austin; 

Richard  Larson,  M.I.T.; 

Alexander  Levis,  M.I.T.; 

John  Little,  M.I.T. ; 

Lola  Lopes,  University  of  Wisconsin; 

Thomas  Magnanti,  M.I.T. ; 

Trudi  Miller,  National  Science  Foundation; 

Laurence  Moore,  Virginia  Polytecnic  Institute  and  State  University; 

William  Pierskalla,  University  of  Pennsylvania; 

Andrew  Sage,  University  of  Virginia; 

Subrata  Sen,  Yale  University; 

Randolph  Simpson,  Office  of  Naval  Research; 

Paul  Slovic,  Decision  Research; 

Gerald  Thompson,  Carnegie -Mellon  University; 

Robert  Thrall,  Rice  University; 

Edison  Tse ,  Stanford  University; 

Willard  Vaughn,  Office  of  Naval  Research; 

John  Warfield;  Andrew  Whinston,  Purdue  University; 

Robert  Winkler,  Duke  University; 

Detlof  von  Winterfeldt,  University  of  Southern  California; 

Lofti  Zadeh,  University  of  California,  Berkeley. 
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APPENDIX  II 


STATEMENT  OF  THE  NATIONAL  SCIENCE  FOUNDATION  SUBPANEL  FOR  THE  DECISION  AND 

MANAGEMENT  SCIENCE  PROGRAM 


December  1983 


Members  of  the  Subpanel  for  the  Decision  and  Management  Science 
agree  that  the  Program  should  emphasize  basic  research  to  develop  a 
theoretical  and  empirical  science  of  managerial  and  operational  processes. 
Proposals  that  explore  such  common  mange rial  processes  as  planning,  control, 
selection,  monitoring  and  evaluation;  and  such  common  operational  processes  as 
congestion,  distribution,  screening,  and  market  responses,  are  encouraged. 
The  panel  is  particularly  interested  in  developing  a  body  of  knowledge  that 
incorporates  the  social  and  behavioral  aspects  of  these  processes. 

In  the  conduct  of  research,  processes  will  typically  be 
characterized  by  mathematical,  logical,  and  statistical  models.   These  models 
will  be  derived  from  empirical  observation,  or  from  theory  that  is  subject  to 
empirical  verification.   Empirical  analyses  should  be  pursued  in  some 
operational  context,  but  the  emphasis  should  be  on  theories,  findings  and 
methods  that  are  generalizable  to  other  contexts. 

Thus,  the  body  of  research  supported  by  the  program  should  possess 
generality,  be  based  on  empirical  observation  or  be  subject  to  empirical 
validation,  and  incorporate  social  and  behavioural  aspects.   Processes  should 
be  characterized  by  models  that  are  tested  in  operational  contexts.   Even 
though  an  individual  project  may  not  have  all  these  charsacteristics ,  its 
evolution  toward  this  end  must  be  clear. 
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lUTlD 


THE   UrSJIVEPSITY   CF  TEXAS   AT   DALLA; 

eOX  830666      RICHARDSON.  TEXAS  75CC:-  0656       (2-<.  590-2744 


The  UNIV=HStTY  Of  TEXAS  fifTEW 

EUGENE  MCOeHMCTT  PROf  ESSOP  Or"  MANAGEMENT 


July  5,  19S4 


Dr.  Otto  N.  Larsen 
Senior  Associate  for  the 

Social  and  Behavioral  Sciences 
National  Science  Foundation 
Washington,  DC  2C550 

Dear  Dr.  Larsen: 

V/e  regret  that  you  were  unable  to  attend  our  workshop  on  research 
priorities  for  decision  and  management  science  in  Dallas.  However, 
through  various  media,  we  will  endeavor  to  make  results  available 
for  your  use  in  NSF  planning. 

The  major  workshop  product  will  be  a  statement  about  research  priorities 
that  is  being  compiled  by  Dr.  John  D.  C.  Little,  in-coming  president  of 
The  Institute  of  Management  Sciences  (TIMS)  and  Area  Head  for  the 
Behavioral  and  Policy  Sciences  in  the  Sloan  School  of  Management  at 
M.I.T.  Dr.  Little's  statem.ent  will  be  sent  to  you  as  soon  as  it  is 
released;  it  will  also  be  printed  in  Interfaces,  a  joint  publication 
of  TIMS  and  ORSA.  Parallel  statements  will  appear  in  IEEE  Transactions 
on  Systems,  Man,  and  Cybernetics  and  other  relevant  journals  and  news- 
letters. 

This  letter  conveys  supplementary  recommendations  about  management 
policies  that  are  consistent  with  the  subject  area  priorities.  These 
recomjnendations  from  the  workshop  were  elaborated  through  comments  by 
workshop  participants  and  other  leaders  of  the  DMS  research  community 
on  a  first  draft  of  this  letter.  As  a  former  member  o'f  the  DMS  Advisory 
Panel,  I  know  that  these  suggestions  are  compatible  with  NSF/BBS  policies; 
indeed,  many  have  already  been  adopted  by  DMS. 

The  widely  circulated  Statement  of  the  Panel  for  DMS  (attached)  lies 
behind  both  the  research  priorities  that  Dr.  Little  will  describe  and 
these  recommendations  for  program  management.  The  features  that  program 
management  should  accommodate  are  its  (1)  focus  on  fundamental  or  general 
processes,  (2)  multi-disciplinary  approach,  (3)  emphasis  on  research  in 
operational  contexts,  (4)  reliance  on  large-scale  projects,  and  (5) 
utilization  of  the  expanding  capabilities  of  computers.  Specific  recom- 
mendations follow  in  these  categories. 


AN  EQUAL  OPPOnTUMITV/A'FlBMATIVC  ACTlOM  UNIVERSITY 
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Focus  on  Fundamental  or  General  Processes 

Recommendations: 

DMS  should  support  research  on  fundamental  processes,  which 
implies  funding  proposals  that  are  "high  risk"  from  relevant 
disciplinary  perspectives  -  a  couple  of  negative  reviews 
should  not  be  allowed  to  "kill"  an  otherwise  promising 
project. 

Research  should  rely  on  formal,  especially  mathematical, 
models;  and  projects  that  concentrate  on  formal  theory 
should  be  encouraged,  provided  that  they  are  subject  to 
empirical  verification  in  the  long  run. 

Discussion:  Too  many  theories  and  findings  in  decision-related 
fields  can  be  summed  up  by  saying:  "It  depends."  In  part,  this 
lack  of  generality  reflects  the  selection  of  research  projects 
that  will  pay-off  quickly  in  applications  or,  for  basic  research, 
in  publications.  Psychology's  emphasis  on  controlled  laboratory 
experiments  also  deflect  research  away  from  many  questions  that 
are  fundamental  for  DMS.  In  contrast,  the  most  powerful  and 
useful  theories  of  DMS  are  both  abstract  and  generally  appli- 
cable: they  are  mathematical  representations  of  common  processes, 
such  as  diffusion  and  propagation,  selection  and  screening, 
evolution,  and  homeostatic  processes.  Also,  multi-disciplinary 
efforts  in  DMS  are  most  likely  to  succeed  when  disciplinary  per- 
spectives are  translated  into  mathematical  models,  so  that 
assumptions  can  be  debated  in  the  common  language  of  science. 
To  emulate  these  past  successes,  new  research  supported  by  DMS 
should  employ  formal  models  and  address  fundamental  questions, 
even  though  some  reviewers  may  favor  narrowing  inquiries  to 
maximize  internal  validity  and  short-term  pay-off.  Also,  the 
"escape  clause"  in  the  Panel  Statement  should  be  used  to  encourage 
theoretical  research  that  concentrates  on  "first  generation"  (new, 
potentially  high-payoff)  models,  provided  that  these  abstractions 
are  subject  to  empirical  verification  in  the  long  run. 

Multi -Disciplinary  Approach 

Recommendations: 

More  than  disciplinary  programs,  DMS  should  support  conferences 
to  aggregate  and  evaluate  alternative  disciplinary  approaches 
to  fundamental  problems.  Specifically,  (a)  the  DMS  panel 
should  guide  the  planning  of  one  Dallas-type  workshop  per 
year,  and  (b)  the  Program  should  encourage  investigator- 
initiated  conference  proposals. 
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The  DMS  panel  should  also  consider  other  mechanisrr.s  for 
providing  "seed  money"  for  promising  cross-discipl  in^-sry 
efforts,  including  tutorials,  multi-disciplinary  reviews 
of  research,  post-doctoral  fellowships  in  other  disciplines, 
and  cooperation  with  the  Engineering  Directorate  in  support 
of  interdisciplinary  centers. 

Discussion:   Interdisciplinary  departments  for  "management"  have 
emerged  in  universities,  and  their  interest  in  science  (as  well 
as  in  practice)  is  represented  by  TIMS.  Because  generic  manage- 
ment departments  have  student  constituencies  and  control  over 
faculty  tenure,  they  provide  a  promising  university  base  for 
scholarship  in  DMS.  However,  the  focus  of  most  management 
programs  is  on  practice  not  science,  and  related  professional 
associations  (those  grounded  on  electrical  engineering  and  the 
social  and  behavioral  sciences)  have  little  overlap  with  TIMS, 
which  is  grounded  on  operations  research  and  management  science 
(OR/MS).  Thus,  especially  in  the  short  run,  DMS  should  encourage 
cross-fertilization  among  disciplines. 

Emphasis  on  Research  in  Operational  Contexts 

Recommendations: 

DMS  should  encourage  projects  that  develop  and  verify  general 
theories  in  real-world  contexts,  especially  in  collaboration 
with  practitioners.  University/industry  projects  to  collect, 
update,  and  redefine  data  elements  for  calibrating  and  vali- 
dating fundamental  theories  should  have  high  priority. 

Discussion:  On  the  face  of  it,  the  DMS  Panel's  emphasis  on  both 
generality  and  operational  contexts  seems  contradictory:  How  can 
models  be  abstract  and  general  when  organizations  are  diverse  and 
solutions  are  tailored  to  specific  conditions?  In  fact,  the  flag- 
ship theories  (applications)  of  DMS  are  known  to  possess  generality 
and  to  highlight  the  critical  features  of  apparently  diverse 
problems  in  varied  settings.  Generality  and  applicability  comoine 
when  abstract  models  capture  fundamental  processes.  Moreover,  a 
current  weakness  of  research  in  DMS  is  its  focus  on  "fourth  genera- 
tion" problems,  or  refining  existing  models,  to  the  neglect  of 
"first  generation"  problems,  or  developing  new  models.  Thus  DMS 
should  encourage  scientists  to  study  phenomena  in  real  world  contexts. 
Collaboration  between  practitioners  and  scientists  in  addressing 
pressing  issues,  such  as  the  opportunities  (and  difficulties)  of 
technological  advancement,  will  facilitate  the  design,  estimation, 
and  validation  of  new  models.  Of  special  importance  are  practitioner/ 
university  efforts  to  collect,  update,  and  redefine  data  elements 
as  models  evolve  over  time. 
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Reliance  on  Large  Scale  Projects 

Recommendations: 

DMS  should  encourage  but  not  pay  for  the  full  cost  of  large 
scale,  multi-disciplinary,  collaborative  projects  between 
practitioners  and  scholars  in  operational  contexts.  Oppor- 
tunities for  joint  funding  with  other  Federal  agencies,  NSF 
programs  for  university/industry  cooperation,  and  private 
sector  organizations  should  be  explored.  For  especially 
promising  proposals,  DMS  should  also  invest  considerably 
more  than  the  average  award  amount  in  multi-investigator, 
empirical  research. 

Discussion:  Past  breakthroughts  in  DMS  have  generally  involved 
multi-disciplinary  teams  working  with  practitioners  in  operational 
contexts  to  solve  real  problems,  often  with  funding  from  the 
Defense  Department.  Large  scale  multi-disciplinary  research  on 
fundamental  aspects  of  defense-related  problems  continues  to 
receive  support,  primarily  through  the  Office  of  Naval  Research 
(ONR).  Parellel  efforts  to  address  markets,  strategic  and  long 
range  planning,  management  controls,  distributed  decision-making, 
personnel  requirements,  and  operational  processes  in  domestic 
settings  should  be  encouraged  by  DMS.  However,  DMS  should  seek 
joint  funding  for  these  multi-purpose,  multi-actor  efforts,  not 
pay  full  costs.  Also,  because  most  successful  models  in  DMS 
apply  equally  well  to  military  and  domestic  processes,  close 
coordination  should  be  maintained  with  ONR  and  other  Defense  Depart- 
ment programs  that  support  fundamental  research. 

Utilization  of  the  Expanding  Capabilities  of  Computers 

Recommendations: 

In  collaboration  with  related  NSF  programs,  DMS  should  encourage 
research  on  how  humans  can  best  employ  the  unfolding  capabilities 
of  computers.  Relevant  contexts  range  from  using  supercomputers 
to  find  numerical  solutions  for  operational  problems  (machine- 
dominated  decision  making)  to  the  use  of  expert  systems  inter- 
actively in  decision  support  systems  (man-dominated  decision 
making). 

Discussion:  The  decision-related  sciences  increasingly  employ  and 
address  man-made  systems.  Agreement  with  NSF  on  the  proper  approach 
to  constructing,  validating,  and  applying  scientific  theories  about 
man-made  systems  has  yet  to  be  achieved.  In  the  interim,  the 
interests  of  leaders  in  the  three  principal  sub-fields  of  DMS  (i.e., 
OR/MS,  electrical  engineering,  and  the  social  and  behavioral  sciences) 
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are  converging:  all  want  to  conduct  multi-disciplinary  research 
on  human/maching  combinations  for  decision  making.   Especially  in 
this  subject  area,  behavioral  science  methods  that  stress  internal 
validity  should  be  questioned  and  theory-driven  research  should  be 
encouraged. 

As  I  indicated  above,  these  recommendations  for  DMS  Program  management 
were  developed  in  consultation  with  representatives  of  the  three  broach 
disciplinary  constituencies  of  DMS  --  OR/MS,  electrical  engineering,  and 
the  social  and  behavioral  sciences.  I  would  be  happy  to  bring  leaders 
of  these  communities  who  work  in  the  field  of  DMS  together  with  you  and 
Others  to  talk  about  program  management  and  research  priorities. 


Sincerely, 


Frank  M.  Bass 

The  University  of  Texas  System 

Eugene  McDermott  Professor  of  Management 

/eas 
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Ask  Not  What  Psychology  Can  Do  for  DMS; 
Ask  What  DMS  Can  Do  for  Psychology 


Kenneth  R.  Hammond 

Center  for  Research  on  Judgment  and  Policy 
Institute  of  Cognitive  Science 
University  of  Colorado 
Boulder,  Colorado 


Decision  and  Management  Science  Workshop 
The  University  of  Texas  at  Dallas 
April  26,  27,  1984 


Ask  Not  What  Psychology  Can  Do  for  DMS; 
Ask  What  DMS  Can  Do  for  Psychology 


According  to  the  directive  from  Alfred  Blumstein  each  panelist  "is 
expected  to  put  on  the  table  some  specific  suggestions  of  high  payoff 
opportunities  whereby  his  discipline  can  contribute  to  the  DKS  program 
objectives."  I  will  try  to  meet  that  expectation  by  first  indicating  what  I 
believe  would  be  the  current  consensus  view  of  psychologists  regarding 
potential  contributions  from  psychology  to  the  DMS  program.  Because  of  my 
personal  conviction  that  the  NSF/DMS  program  can  do  more  for  psychology 
than  psychology  can  do  for  it,  in  the  second  part  of  my  paper  I  will 
indicate  what  I  believe  the  contributions  to  psychology  from  DMS  might  be. 

Potential  Contributions  from  Psychology  (Consensus  View) 

From  the  Literature  of  Indirect  Comparisons 

The  most  salient  line  of  research  in  the  field  of  judgment  and 
decision  making  during  the  1970's  involved  the  comparison  between  a 
person's  efforts  to  make  rational  choices  with  the  choices  produced  by  a 
rational  model,  for  example,  Bayes '  Theorem.  This  line  of  research 
addressed  the  question  of  the  rationality  of  information  processing, 
particularly  with  regard  to  choice  behavior. 

The  main  result  of  this  research  (carried  out  largely  by  Daniel 
Kahneman,  Paul  Slovic,  Amos  Tversky,  and  their  colleagues)  was  to  demolish 
the  descriptive  validity  of  the  expected  utility  model,  apparently  much  to 
the  surprise  of  economists,  philosophers,  and  some  psychologists.  These 
results  have  received  wide  recognition  among  all  three  disciplines.  For 
example,  Kenneth  Arrow  (1982),  in  a  paper  entitled  "Risk  Perception  in 
Psychology  and  Economics"  notes  that 
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The  rationality  or  irrationality  of  choice  has  become  a  leading 

interest   of   the   branch   of  psychology  called  'cognitive 

psychology'.  •  .  .in  the  last  twenty  years  it  has  become  a  major 

field  of  psychological  research,  in  contrast  to  earlier  work 

which  tended  to  emphasize  either  the  role  of  emotions  or 

mechanistic  models  for  learning,   (p.  1) 

Arrow  then  Indicates  that   "there   has   been   renewed   testing   of 

expected-utility  theory;   one  striking  result  has  been  the  series  of 

stunning  experiments  on  the  so-called  'preference  reversal'  phenomenon  by 

Lichtenstein  and  Slovic  (1971)"  (p.   2).   After  drawing  a  series  of 

analogies  with  various  phenomena  in  the  field  of  economics,  Arrow  points  to 

the  work  of  Tversky  and  Kahneman  (1974,  1981)  in  which  he  notes  that 

"several  heuristic  devices  [have  been  discovered]  by  which  individuals  form 

cognitive  judgements  and  while  each  has  useful  properties,  each  can  also 

lead  to  biases  in  judgement"  (p.  5).  He  then  shows  how  a  specific  example 

from  their  work  "typifies  very  precisely  the  excessive  reaction  to  current 

information  which  seems  to  characterize  all  the  securities  and  futures 

markets"  (p.   5).   Arrow  concludes  his  paper  by  saying,  "I  hope  to  have 

made  a  case  for  the  proposition  that  an  important  class  of  intertemporal 

markets  shows  systematic  deviations  from  individual  rational  behavior  and 

that  these  deviations  are  consonant  with  evidence  from  very  different 

sources  collected  by  psychologists"  (p.  8).  Attention  of  this  sort  from  a 

Nobel  Laureate  in  the  discipline  of  economics  to  research  in  psychology 

certainly  falls  in  the  category  of  a  rare  event. 
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A  second  consequence  of  the  work  on  indirect  comparisons  has  been  to 
encourage  economists  to  turn  to  laboratory  experiments  (see  especially 
Smith,  1982;  Plott,  1979).  Apparently  inspired  by  the  work  of  Kahneman 
and  Tversky  there  now  appears  to  be  an  established  trend  among  economists 
toward  the  use  of  the  general  laboratory  research  paradigm  conmonl  y  used  by 
psychologists. 

Research  opportunities.  The  rich  assortment  of  findings  produced  by 
psychologists  regarding  the  irrationality  of  choice  behavior  provides  many 
research  opportunities  of  which  I  list  only  four  general  ones: 

1.  Discover  which  of  the  heuristics  identified  by  psychologists  will 
be  used  when,  why,  how,  in  which  operational  context. 

2.  Discover  the  costs  of  nonrational  choice  behavior  in  various 
operational  contexts. 

3.  Discover  remedies  for  nonrational  choice  behavior  in  operational 
contexts. 

4.  Discover  the  benefits  of  rational  choice  behavior,  if  it  can  be 
induced. 

From  the  Literature  of  Direct  Comparisons 

A  second  line  of  research,  somewhat  older  than  the  one  above,  directly 
compares  the  judgments  of  a  person  with  an  empirical  criterion,  and  thus 
evaluates  the  empirical  accuracy  of  judgments  and/or  predictions.  There 
are  two  main  results  from  this  research  effort: 
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1.  Simple  linear  models  predict  most  task  outcomes  very  accurately 
(better  than  expert's  judgment). 

2.  Simple  additive  (or  weighted  sum)  models  predict  judgments  very 
accurately. 

The  main  conclusion  from  this  research  is  that  linear  models,  rather  than  a 
person,  should  be  given  the  information  necessary  for  a  prediction,  even  if 
the  person  is  an  expert. 

•  Research  opportunities.  This  line  of  research  also  provides  many 
opportunities  for  research  within  the  Decision  and  Management  Science 
context.     For  example, 

1.  Discover  when,  how,  why,  which  operational  contexts  make  the 
simple  linear  model   inappropriate. 

2.  Discover  how  to  improve  the  accuracy  of  judgments  (especially 
among  experts)  in  operational  contexts. 

3.  Make  direct  comparisons  of  the  accuracy  of  analytical, 
quasirational ,  and  intuitive  judgments  produced  by  experts  in 
various  operational   contexts. 

4.  Discover  the  differential  costs  and  benefits  of  the  use  of  these 
modes  of  cognition  in  operational   contexts. 

From  the  Literature  of  Problem  Solving,  Memory, 
and  Text  Comprehension  Research 

The  main  topics  considered  by  researchers  in  this  field  include;       (a) 
the     sequences     of    heuristics  used  in  problem  solving,  and  (b)  the  role  of 
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memory  in  the  course  of  problem  solving,  and  the  ability  to  comprehend 
written  material.  One  main  result  is  that  the  course  of  problem  solving 
can  be  simulated  by  a  computer  program.  One  main  conclusion  is  that  people 
(especially  experts)  store  large  amounts  of  information  and  use  it 
effectively,  often  in  the  form  of  "shortcuts." 

This  field  of  research  has  also  produced  a  rich  variety  of  research 
opportunities  for  DMS.  One  salient  opportunity  is  provided  by  the  strange 
contradiction  between  the  results  achieved  by  judgment  and  decision  making 
researchers  and  problem  solving  researchers  regarding  experts.  That  is, 
the  judgment  and  decision  making  researchers  generally  find  that  experts 
fail,  whereas  the  problem  solving  researchers  generally  find  that  experts 
are  superior  problem  solvers.  What  causes  the  difference  between  these  two 
conclusions?  Where  does  the  truth  lie?  It  is  hard  to  think  of  a  more 
important  problem  for  Decision  and  Management  Science. 

Results  Common  to  All  Three  Research  Areas  , 

All  three  research  areas  have  addressed  the  problem  of  the  generality 
of  results  over  task  conditions.  The  main  result  is  clear:  Results  from 
psychological  research  are  highly  task  dependent;  results  do  not 
generalize  over  tasks.  That  leads  to  a  very  important  conclusion  reached 
by  all  reviewers  in  the  Annual  Review  of  Psychology  since  1977,  namely 
"caveat  emptor"!  Researchers  in  the  field  of  Decision  and  Management 
Science  who  wish  to  build  upon  the  work  of  psychologists  must  be  cautious; 
the  result  that  indicates  that  people  do  this  or  that  is,  in  all 
likelihood,  restricted  to  the  laboratory  circumstances  in  which  it  was 
generated.  For  example,  almost  every  result  from  psychological  research  is 
information  restricted  and  time-restricted.  If  your  "operational  context" 
is  one  in  which  the  actors  will  want  to  seek  information  and  have  the  time 
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(hours,  days  or  weeks)  to  do  it,  then  the  results  that  imply  that  people 
improperly  process  information  are  not  likely  to  apply;  at  least,  the 
burden  of  proof  lies  with  those  who  wish  to  apply  the  results. 

The  primary  research  opportunity  here  is  to  find  the  road  to  the 
discovery  of  enduring  generalizations.  Can  the  Decision  and  Management 
Science  Program  of  the  National  Science  Foundation  help  with  this  problem? 
I  hope  so.  And  that  hope  brings  me  to  my  personal  view  of  the 
contributions  of  Decision  and  Management  Science  Program  to  psychology. 

Contributions  to  Psychology  from  DMS 

First  let  me  cite  the  DMS  policy  statement: 

Thus,  the  body  of  research  supported  by  the  program  should 
possess  generality,  be  based  on  empirical  validation,  and 
incorporate  social  and  behavioral  aspects.  Processes  should  be 
characterized  by  models  that  are  tested  in  operational  contexts. 
Even  though  an  individual  project  may  not  have  all  these 
characteristics,  its  evolution  toward  this  end  must  be  clear. 

This  policy  statement  raises  a  number  of  important  questions. 
Consider  first  the  requirement  of  "generality."  Does  psychological  research 
produce  results  that  are  general  over  a  range  of  tasks?  The  answer  to  this 
inust  be  "no."  Can  psychological  research  produce  generalizable  results? 
The  answer  to  this,  I  believe,  must  also  be  "no."  Why  not?  Because 
psychologists  do  what  they  can  do  within  a  traditional  research  paradigm. 
That  Is,  they  must  carry  out  (literally)  cheap  research  projects  that 
Involve  minimal  information  displays,  free  subject  time,  problems  cut  to 
fit  the  college  lecture  hour,  problems  tailored  to  a  methodology 
(uncritically   inherited   from   agriculture)  that  inherently  prevents 
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generalization,  and  the  fact  that  promotion  within  psychology  departments 
is  based  on  quantity  of  publication,  a  practice  which  encourages  the 
maintenance  of  cheap  research. 

A  second  set  of  questions  is  raised  by  the  prerequisite  of 
"operational  context."  Does  the  typical  psychology  laboratory  study 
(complex  problems  cut  to  fit  the  sophomores  50-minute  hour)  meet  the 
"operational  context"  criterion?  No.  Is  there  "evolution"  toward  research 
in  an  "operational  context"?  No;  indeed,  the  trend  is  in  the  reverse 
direction  as  may  be  seen  in  the  enthusiasm  expressed  by  economists  for 
laboratory  research  in  the  style  of  psychologists.  My  prediction  here  is 
that  there  will  be  an  uncritical  examination  of  the  nature  of  psychological 
research,  and  that  as  a  result,  economists  will  reinvent  the  square  wheel. 
.1  have  considerable  enthusiasm  for  Vernon  Smith's  (1982)  suggestion 
regarding  what  he  calls  "parallelism,"  by  which  he  (roughly)  means  the 
effort  to  find  convergent  validity  of  results  from  laboratory  and  "real 
world"  research.  But  he  should  be  made  aware  of  the  thorough  treatment  of 
the  concept  of  convergent  validity  by  the  psychologists  Campbell  and  Fiske 
(1959)  who  provide  a  method  for  examining  both  the  convergent  and 
discriminant  validity  of  various  measures. 

Barriers  to  General izable  Research 

In  my  view,  the  following  barriers  to  general izable  research  in 
psychology  exist: 

1.  Most  psychologists  believe  that  research  should  not  be  carried  out 
in  an  operational  setting.  Achieving  rigor  is  more  important  to 
experimental  psychologists  than  achieving  complexity. 
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2.  Most  psychologists  have  a  trained  incapacity  for  research  in  an 
operational  context.  That  is,  most  psychologists  would  not  know 
how  to  do  this  type  of  research  even  if  they  wanted  to. 

3.  Most  psychologists  must  work  in  nonoperational  contexts  (that  is 
the  psychological  laboratory);  they  have  no  choice  if  they  wish 
to  be  promoted. 

As  a  result  of  these  barriers,  results  are  exactly  what  we  would 
expect  them  to  be,  namely,  task  dependent;  enduring  generalizations  cannot 
be  achieved  under  these  conditions.  At  this  point  you  may  be  wondering 
whether  my  personal  views  are  shared  by  other  psychologists.  They  are  in 
fact  shared  by  very  few  psychologists,  but  if  the  number  is  small,  the 
quality  is  very  high.  One  of  psychology's  most  outstanding  scholars,  Paul 
Meehl ,  recently  described  the  course  of  research  efforts  in  psychology  in 
this  way: 

There  is  a  period  of  enthusiasm  about  a  new  theory,  a  period  of 

attempted  application  to  several  fact  domains,  a  period  of 

disillusionment  as  the  negative  data  come  in,   a   growing 

bafflement  about  inconsistent  and  unreplicable  empirical  results, 

multiple  resort  to  ad  hoc  excuses,  and  then  finally  people  just 

lose  interest  in  the  thing  and  pursue  other  endeavors.  (1978,  p. 

807) 

Finally,  Meehl    (1978)  observes  that     "the     enterprise     shows     a     disturbing 

absence     of    that  cumulative  character  that  is  so  impressive  in  disciplines 

like  astronomy,  molecular  biology,  and  genetics"   (p.     807). 
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The  Role  of  DMS 

The  DMS  policy  statement,  if  taken  seriously,  offers  several  research 
opportunities  to  psychologists,  if  DMS  is  indeed  serious  in  its 
requirements  for  general izable  research  in  an  operational  context.  I  offer 
below  five  examples  of  advances  that  psychological  research  will  never  make 
unless  they  are  carried  out  within  the  DMS  policy  statement. 

1.  Understanding  decision  making  over  time 

2.  Understanding  expert  jud^ent 

3.  Understanding  the  tension  between     logical     consistency     ("truth") 
and  accuracy 

4.  Understanding  group  decision  making  among  adults 

5.  Understanding  the  nature  of  causal   inference 

DMS/KSF  can  help  break  the  tradition  of  "knowledge  on  the  cheap"  by 
requiring  research  methods  that  explicitly  address  the  matter  of 
generality,  and  by  providing  the  dollar  support  for  projects  that  will 
incorporate  such  methods.  DMS  can  also  help  break  the  tradition  of 
noncumulative  research  findings  by  requiring  research  to  be  done  in  an 
"operational  context"  or  simulation  thereof,  and  by  providing  dollar 
support  for  such  research. 
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HIGH  PAYOFF  OPPORTUNITIES  IN  SYSTEMS  THEORY  FOR  THE  DMS  OBJECTIVES 

By 

Edison  Tse 
Departnent  of  Engineering-Economic  Systems 
Stanford  University 

■ 

Decision  making  is  a  dynamic  multi-stage  iterative  process.  Within 
it  are  sequences  of  "convergent"  and  "divergent"  subprocesses  that  take 
the  decision  maker  from  the  recognition  that  he  has  a  decision  problem 
to  the  implementation  of  his  preferred  choice.  Fig.  1,  shows  a  model  of 
the  decision  process.  The  specific  breakdown  into  different  distinct 
phases  is  not  unique,  but  it  does  allow  us  to  distinguish  the  different 
divergent  and  convergent  processes.  Each  divergence  provides  a  scope 
for  the  following  convergence;  each  convergence  provides  a  focus  for  the 
following  divergence.  Nearly  all  research  in  the  area  of  decision 
theory  has  been  focusing  on  the  choice  proess.  While  this  is  an 
important  process,  a  thorough  understanding  of  decision  making  so  as  to 
allow  us  to  make  "better"  decisions  requires  a  systematic  study  on  the 
overall  dynamic  process  of  decision  making,  not  just  the  choice  proces 
alone. 

The  lack  of  systematic  research  activity  in  the  subprocesses  prior 
to  the  choice  process  may  be  due  to  the  fact  that  no  convenient 
mathematical  tool  is  available  to  allow  us  to  study  them;  whereas  choice 
theory  can  be  formulated  and  studied  under  the  statistical  decision 
theory  or  stochastic  control  theory  framework.  One  can  broadly 
sub-divide  the  decision  process  into  two  main  subprocesses:  generation 
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Fig.  1,  A  multi -stage  process  of  decision  making 
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and  choice.  The  kind  of  activities  that  a  decision  maker  is  involved  in 
each  of  these  two  main  subprocesses  are  substantially  different.   In  the 
generation  process,  the  main  goal  is  to  create  new  alternatives — this  is 
a  divergent  process.  The  main  tools  would  be  association,  reasoning  via 
analogy,  adapting  guidelines  based  on  past  experience,  or  expert 
knowledge.  We  shall  call  these  intelligent  activities.  The  choice 
process,  on  the  other  hand,  requires  focusing  on  the  alternatives  being 
generated,  carefully  studying  the  implications  of  these  alternaties  via 
analysis,  and  ranking  these  alternatives  via  certain  criteria  in  order 
to  recommend  the  final  choice — this  is  a  convergent  process.  The  main 
tools  would  be  a  whole  host  of  mathematical  analyses  including 
statistical  decision  theory,  optimization,  control  and  many  others.  We 
shall  call  these  analytical  activities. 

In  the  past,  we  have  been  mainly  researching  on  the  analytical 
aspect  of  decision  making  because  analytical  tools  have  a  long  history 
of  development;  we  are  very  familiar  with  them  and  we  know  how  to  use 
them  well.  The  study  of  the  intelligent  activity  in  decision  making  has 
emerged  in  the  fields  of  psychology  and  artificial  intelligence 
(commonly  referred  to  as  AI).  The  recent  development  of  rule-based 
expert  systems  within  the  AI  discipline  seems  to  offer  a  new  efficient 
tool  to  allow  us  to  study  the  generation  process  within  decision 
making.  [1]  Recent  developments  in  computer  hardware  will  provide  new 
dimension  in  decision  making  research.  The  ease  in  man-computer 
interaction  through  advancements  in  interface  technology  will  offer  a 
new  perspective  in  research  methodology.   I  perceive  that  the  high 
payoff  and  exciting  opportunities  are  surrounding  the  integration  of  new 
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technological  development  in  expert  systems  and  computer  hardware  with 
the  familiar  analytical  methods  to  provide  a  coherent  model  of  the 
decision  making  process  that  can  be  studied  and  validated  in  real 
operational  settings.  Moreover,  such  investigations  will  lead  to  the 
improvement  on  the  decision  process.  In  the  following.  I  shall  be  a 
little  more  specific  on  the  directions  which  I  believe  will  lead  to  my 
vision  beyond  that  long  tunnel. 

A  new  framework  needs  to  be  developed  to  allow  us  to  handle  the 
generation  and  choice  paradigm — in  particular,  an  uncertainty 
representation  framework  which  is  flexible  enough  to  allow  us  to  deal 
with  ignorance  as  would  be  introduced  in  the  generation  process. 
Consider  the  example  that  we  hear  the  noise  of  the  jet  engine  which 
allows  us  to  form  a  subjective  probability  of  0.9  that  a  jet  plane  has 
just  flown  by.  The  necessity  to  determine  what  type  of  a  jet  plane  it 
is  does  not  seem  to  be  important.  However,  as  the  decision  process 
unfolds,  it  may  become  apparant  later  that  the  knowledge  of  what  type  of 
plane,  in  particular  whether  it  is  DC-10  or  747,  that  had  flown  by  is 
crucial  in  order  to  make  the  right  choice.  If  we  now  generate  the 
hypothesis  space  from 

H  =  {plane,  others) 
to 

Hp   =  (747,  DC-10,  other  jet  plane,  others} 
then  how  should  the  probability  measure  ProbCplane}  =  0.9  , 
Prob{others}iJ=0.1  "propagate"  to  Hp  .  Basic  probability  theory 
requires  us  to  have  knowledge  on  conditional  probability 

Prob{747/plane},  etc. 
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However,  this  may  be  too  much  to  require  since  such  knowledge  may  not 
come  by  easily  without  detailed  statistical  information  on  the  number  of 
planes  in  each  type,  flight  schedules  of  all  the  planes,  weather 
conditions  everywhere  that  may  cause  delay,  etc.  To  reflect  ignorance, 
we  may   want  to  assign  0.9  to  the  set  {747,  DC-10,  other  jet  planes} 
which  Is  a  subset  of  Hp  .   If  additional  evidence  can  be  collected 
that  can  reduce  our  ignorance,  we  would  certainly  do  so;  however,  if  we 
are  forced  to  make  a  choice,  then  we  have  to  perform  reasoning  based  on 
the  uncertainty  representation  on  Hp  which  incorporates  the  notion 
of  ignorance.  Now  the  uncertainty  representation  on  Hp  is  not 
Bayesian,  but  rather  a  representation  introduced  by  Dernpster-Shafer. 
[23  Another  uncertainty  representation  framework  which  accounts  for 
semantic  ambiguity  is  the  possibility  model  of  Zadeh.  [3]  This  model  has 
recently  been  widely  discussed  in  the  literature,  but  the  role  of  such 
representation  to  decision  making  is  not  well  studied. 

While  statistical  decision  theory  is  an  appropriate  theory  based 
Dn  Bayesian  representation  of  uncertainty,  new  decision  theories  need  to 
be  developed  based  on  different  representations  of  uncertainty  which 
incorporates  ignorance  and/or  semantic  ambiguities.  How  do  we 
characterize  risk  under  such  representations  of  uncertainty?  Can  these 
models  account  for  the  seemingly  "irrational"  behavior  of  the  decision 
maker? 

Another  important  research  direction  is  the  development  of  a  new 
tool,  or  a  methodology  that  can  enhance  our  understanding  of  the  dynamic 
iteration  between  generation  and  choice  processes.  No  coherent  approach 
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exists  in  the  literature  because  the  two  processes  call  for  completely 
different  reasoning  procedures.  I  believe  that  such  a  research 
direction  requires  an  integration  of  AI's  expert  systems  technology, 
user  interface  technology,  system  analysis  and  human  interaction.  Just 
like  linear  programming  is  a  very  useful  tool  in  the  study  of  operations 
research,  the  tool  to  be  developed  that  will  be  useful  in  the  study  of 
the  decision  process  is  an  interactive  simmulation  and  expert 
environment. 

We  are   all  familiar  with  simulation  models  where  a  set  of 
differential  or  difference  equations  is  used  to  model  the  evolution  of 
dynamic  processes.  With  the  advance  in  interactive  computing,  we  can 
perceive  a  simulation  environment  where  the  equations  used  in  the 
simulation  are  parameterized.  Then  the  user  can  easily  perform  "what 
if"  experiments  to  study  the  response  of  the  systems  dynamic  to 
parameter  variations  by  changing  the  parameter  values  in  an  interactive 
manner.  With  the  advancement  in  expert  systems  technology,  we  can 
introduce  rules  for  selection  of  the  equation  structure  to  be 
simulated.  [4]  For  example,  we  can  impose  the  following  rules  based  on 
empirical  evidences: 

•  An  individual  has  vision  ■*   options  will  be  generated  to  increase 
chance  of  fulfillment  of  vision 

•  An  individual  has  little  responsibility  ■>  less  concerned  with 
risk 

Flow,  combining  these  rules 
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•  An  individual  has  a  certain  vision  +  has  little  responsibility  -»• 
generating  options  that  can  allow  him  to  increase  his  chance  of 
goal  fulfillment  while  not  much  concerning  about  the  risk  he  will 
have  to  undertake. 
The  conclusion  may  be  represented  by  a  model 

max{Prob{goal  fulfillment}/resource  and  timing  constraints} 
or  a  particular  analytical  model  which  can  be  represented  by  a  set  of 
dynamic  difference/differential  equations. 

The  interactive  simulation  and  expert  environment  would  allow  the 
user  to  change  rules,  change  model  and  parameters  interactively  during 
the  simulation.  Explanation  capability  is  provided  to  allow  the  user  to 
trace  through  the  rationale  for  using  certain  models  in  the  simulation. 
This  establishes  a  dialogue  between  a  user  and  the  computer.  With  the 
computer  taking  care  of  the  analytical  manipulation,  the  user  can 
experiment  with  the  implications  of  new  options  that  will  induce  changes 
in  model  structure  in  the  analysis  phase.  The  simulation  environment 
can  also  be  used  to  help  us  to  discover  new  issues  that  may  have  been 
overlooked,  which  will  redirect  our  search  for  new  options,  or 
modification  of  our  old  options  in  order  to  address  such  issues. 
Therefore,  I  perceive  that  such  a  tool  can  be  extremely  useful  in 
developing  a  decision  making  model  in  order  to  obtain  deeper 
understanding  of  the  process,  [5]  as  well  as  an  Integral  part  of  a 
man-machine  decision  system  that  will  enhance  the  process  of  decision 
making. 
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Analysis  of  non-classical  dynamic  systems  which  are  representable 
by  a  mix  of  rule  base  and  differential/difference  equations  triggered  by 
discrete  events  and/or  deliberate  action  can  also  bring  deeper 
understanding  to  the  decision  process.  The  recent  study  on  Discrete 
Event  Dynamic  Systems  is  an  example  of  such  types.  [6]  The  research  was 
motivated  by  a  large  class  of  dynamic  systems  which  are  driven  by 
occurence  of  discrete  events — many  opportunity  driven  decision  making 
processes  can  be  characterized  as  discrete  event  dynamic  processes. 
Studies  of  these  types  of  systems  have  been  done  in  the  context  of 
queueing  network.  The  application  of  it  to  the  context  of  decision 
making  would  be  worthwhile. 

Another  exciting  research  area  is  the  strategic  aspect  of  decision 
making  in  a  "manipulative"  environment.  [7]  The  basic  notion  is  that 
thedecision  making  of  an  individual  is  done  within  an  environment  where 
there  are  many  other  decision  makers  who  act  simultaneously  and/or 
responsively.  However,  there  are  constraints,  which  can  be  natural  or 
artificial,  that  limit  the  options  of  each  decision  maker.  In  a  stable 
environment,  we  can  reformulate  it  as  a  game  model.  In  realistic 
situation,  such  environment  is  subject  to  change  even  though  the  time 
constant  of  such  change  is  very  long.  Many  times,  the  change  is  due  to 
strategic  behavior  of  decision  makers  who  consciously  seek  ways  to 
improve  their  strategic  positions  by  manipulating  the  environment  such 
that  the  resulting  "rules  of  the  game"  are  to  their  favor.  Examples  are 
union  movement,  stockpiling,  emerging  companies  and  many  others.   If  one 
examines  the  strategic  behavior  of  decision  makers,  one  may  find  that 
the  awareness  of  other  players  plays  an  important  role.  The  concept  of 
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asymmetric  relationships  which  leads  to  a  limitation  of  one's  options  is 
the  key  to  such  analysis.  Applying  a  tool  like  the  interactive 
simulation  and  expert  environment  to  the  study  of  strategic  behaivor 
will  provide  further  insight  in  the  science  of  decision  making. 

As  an  example  consider  a  manufacturing  company  which  is 
manufacturing  consumer  goods  in  a  growing  market.  The  manufacturing  of 
the  goods  depends  crucially  on  the  availability  of  certain  input 
material  which  is  produced  by  a  handful  number  of  firms. 

Let  the  production  function  be  represented  by 

\  =  f(.I  Qit)  (1) 

where  Q   is  the  output  production  at  time  t  ,  Q-^  is  the  quantity 
of  supply  from  the  i^"  supplier  at  time  t  and  X  =  number  of 
suppliers  .  With  production  cost  given  by  C[q. )  ,  and  different 
supply  prices  P..  ,  i=l»...A  ,  profit  at  time  t  is 

where  P.   is  the  price  and  Q.   is  the  actual  quantity  of  goods  sold 
t  t 

at  time  t  . 

At  the  beginning  of  period  t  ,  the  firm  secures  a  standing  order 
of  Q^  for  the  goods,  and  if  the  production  Q   is  lower  than  Q  , 
a  measure  of  reputation  of  the  company  to  be  able  to  deliver,  R.  ,  is 
"modified"  via  a  model 
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s  (      0        if 

(  X»x     if 


if  x>0 
x<0 


(3) 


<3?  M     _  (4) 


The  actual   quantity  sold  is  modeled  as 

Qt  +  MftXOj-Qj)     ;  0  <  ♦(Pj^)  <  1 

We  further  assume  that 

This  reflects  that,  due  to  expectation  of  consumers,  capability  of 
obtaining  standing  orders  depends  on  historical  "average"  delivery 
performance. 

Given  the  profit  model  described  by  (l)-(5),  and  if  {Q  ..}  are 
within  the  "control"  of  the  company,  then  the  rational  decision  maker 
will  always  buy  from  the  supplier  who  has  the  lowest  price;  and 
depending  on  the  objective  function  defined  in  terms  of  total  discounted 
profit,  an  "optimal"  purchasing  plan  can  be  determined  for  a  specified 
time  period.  However,  in  realistic  situations,  (Q..)  are  not 
controlled  by  the  company.  This  may  be  due  to  the  fact  that  supply  is 
limited  and  there  is  more  than  one  company  that  wants  to  buy  the  same 
material,  either  to  produce  the  same  product  or  to  produce  a  different 
product.  Thus,  the  company  should  conceive  the  situation  where  it  is 
imbedded  in  an  environment  having  different  decision  makers  engaging  in 
negotiation  for  allocation  of  supply. 

A  strategic  move  of  the  company  is  to  engage  in  a  contractual 
relationship  with  one  or  more  suppliers  in  order  to  ensure  a  certain 
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level  of  supplier  so  as  to  maintain  a  certain  level  of  reputation 
measure  R.  .  This  can  be  achieved  in  different  ways:  e.g.  merging 
(buying  all  or  a  fraction  of  a  big  supplier),  diversification 
(maintaining  business  relationships  with  more  than  one  supplier, 
sometimes  paying  a  premium  to  maintain  a  relationship),  exclusivity 
relationship,  and  many  others.  For  each  of  the  potential  strategies  to 
be  adopted,  the  company  will  result  a  profit  model  different  from  the 
one  represented  by  (l)-(5)  in  that  the  cost  structure  is  different,  the 
models  for  Q.  and  Q^  are  different  due  to  consumer's  expectation. 
Moreover,  the  plausible  variations  in  supply  from  each  supplier  will  be 
different  depending  on  the  specific  strategy  adopted. 

The  interactive  simulation  and  expert  environment  can  be  a  useful 
operational  tool  in  studying  the  effectiveness  of  a  specific  strategy  to 
be  adopted.  With  a  different  strategy,  one  can  select  an  appropriate 
profit  model  for  simulation.  Rules  can  be  developed  as  guides  to  which 
model  structure  to  use  as  well  as  what  "range"  of  sensitivity  studies  to 
be  carried  out  under  a  specific  strategic  option.  The  environment  also 
allows  experimentation  with  many  different  modeling  representations  (one 
may  want  to  change  the  reputation  model,  actual  sale  model,  etc.) 
together  with  many  analytical  studies  based  on  each  representation. 
Such  studies  will  not  only  shed  light  on  the  understanding  of  strategic 
behavior,  they  will  also  enhance  the  process  of  strategic  decision 
making. 
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NOTES 

[1]  "Artificial  Intelligence  (AI)  is  the  part  of  computer  science 
concerned  with  designing  intelligent  computer  systems,  that  is 
systems  that  exhibit  the  characteristics  we  associate  with 
intelligence  in  human  behavior— understanding  language,  learning 
solving  problems,  and  so  on"  quoted  from  The  Handbook  of  Artificial 

Intelligence,  Vol.  1,  edited  by  Arron  Barr  and  Edward  A. ' 

Feigenbaum,  published  by  William  Kaufmann,  Inc.,  1981.  'Expert 
systems,  or  knowledge  engineering,  has  emerged  as  specific 
applications  of  AI  techniques.  Expert  systems  can  be  viewed  as 
intermediaries  between  experts,  who  interact  with  the  systems  in 
"knowledge  acquisition"  mode,  and  users  who  interact  with  the 
systems  in  "consultation"  mode.  Expert  systems  also  provide 
explanation  capability,  both  to  make  the  consultation  more 
acceptable  to  the  user  and  to  help  the  expert  find  errors  in  the 
system's  reasoning  when  they  occur. 

[2]  Dempster's  uncertainty  representation  model  was  developed  in  the 
paper  by  Dempster,  A. P.,  "Upper  and  Lower  Probabilities  Induced  by 
Multivalued  Mapping,"  Ann.  Math.  Stdatist.  38,  1967,  pp.  325-339. 
A  formal  mathematical  representation  of  the  model  and  further 
existensions  of  Dempster's  model  were  advanced  by  G.  Shafer,  A 
Mathematical  Theory  of  Evidence.  Princeton  University  Press*  T976. 
i)uch  representation  allows  the  treatment  of  ignorance  through 
refinement. 

[3]  L.  A.  Zadeh  developed  the  possibility  theory  in  representing 

uncertainty  that  may  contain  semantic  ambiguity.  This  discussion 
can  be  found  in  L.A.  Zadeh,  "Fuzzy  sets  as  a  basis  for  theory  of 
possibility,"  Fuzzy  Sets  and  Systems  1.  1978,  pp.  3-28;  L.A.  Zadeh, 
Possibility  Theory  and  bott  Data  Analysis,"  Mathematical  Frontiers 

of  Social  and  Policy  Sciences.  L.  Cobb  and  R.M.  Ihrall,  eds.  • 

Boulder:  Westview  Press,  1981.  ' 

[4]  A  Simulation  and  Expert  Environment  (SEE)  is  now  being  developed 
within  the  Decision  Systems  Laboratory,  Department  of  Engineering- 
Economic  Systems,  Stanford  University.  SEE  is  an  environment  in 
which  one  can  describe  problems  both  by  numerical  simulation  models 
andby  encoding  expert  knowledge  about  the  problem.  The 
environment  had  been  used  to  study  a  simple  team  model  where  the 
emphasis  was  on  the  dynamic  behavior  of  team  cooperation  in  the 
situation  where  team  members  may  have  biased  belief  about  their 
contribution  to  team  performance.  See  P.  Lounamaa  and  E.  Tse, 
"Dynamic  Model  of  Team  Behavior."  Proc:  American  Control 
Conference.  San  Francisco,  CA  1983,  1340-1345. 

[5]  Research  on  the  use  of  SEE  in  studying  organization  and  oligopoly 
problems,  especially  the  influence  of  risk  behavior  in  implicit  or 
explicit  cooporation  among  agents,  is  carried  out  by  P.  Lounamaa 
under  the  supervision  of  E.  Tse  and  J.  March  at  Stanford 
University. 
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[6]  Discrete  event  dynamic  systems  is  an  active  research  activity 
headed  by  Y.C.  Ho  at  Harvard  University.  See  Y.C.  Ho  and  C. 
Cassadras,  "A  New  Approach  to  the  Analysis  of  Discrete  Event 
Dynamic  Systems,"  Automatica.  Vol.  19,  #2,  pp.  149-167,  1983.  A 
forth  coming  paper  by  Y.C.  Ho,  "On  the  Perturbation  Analysis  of 
Discrete  Event  Dynamic  Systems,"  Journal  of  Optimization  Theory  and 
Application,  Special  G.  Leitmann's  Birthday  Volume,  scheduled  to 
appear  in  1985,  gives  a  detailed  account  of  the  research 
activities. 

[7]  A.  Mir  and  E.  Tse  at  Stanford  University  are  doing  research  in 
nodelling  strategic  decision  behavior  in  a  manipulated 
environment.  A  Ph.D.  thesis  by  A.  Mir  supervised  by  E.  Tse,  "The 
Strategic  Implications  of  Varying  Environments:  Aspects  of 
Decision-Making  under  Instability,  (Oct.  1983)  developed  a  logical 
and  systematic  framework  for  strategic  behavior.  Some  of  these 
results  will  be  submitted  for  publication. 


950/*  013 


-13- 


liT    I  IBRAH1E5 


3    "^OfiD    DD3    DbS    flVM 


MAR  24 


BASEW 


Du 


OCT  0  9'ajll 


I  Lib-26-67 


tar    caxM  i^    0^  IplucJc  aov/>CY" 


